Papers with comparative evaluation
VeriMinder: Mitigating Analytical Vulnerabilities in NL2SQL (2025.acl-demo)
Copied to clipboard
| Challenge: | Application systems using natural language interfaces to databases (NLIDBs) have democratized data analysis, but they are not without significant risks. |
| Approach: | They propose an interactive system that detects and mitigates cognitive biases in analytical questions by using contextual semantic mapping frameworks. |
| Outcome: | The proposed system detects and mitigates cognitive biases in analytical questions and generates high-quality, task-specific prompts. |
HiNER: A large Hindi Named Entity Recognition Dataset (2022.lrec-1)
Copied to clipboard
Rudra Murthy, Pallab Bhattacharjee, Rahul Sharnagat, Jyotsana Khatri, Diptesh Kanojia, Pushpak Bhattacharyya
| Challenge: | Named Entity Recognition (NER) is a lowerlevel task that aims to provide class labels like Person, Location, Organisation, Time, and Number to words in free text. |
| Approach: | They propose to use a standard-abiding Hindi NER dataset to analyze the annotations of a class of naming entities in free text. |
| Outcome: | The proposed dataset achieves a weighted F1 score of 88.78 with all the tags and 92.22 when we collapse the tag-set. |
The Ranking Blind Spot: Decision Hijacking in LLM-based Text Ranking (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated strong performance in information retrieval tasks like passage ranking. |
| Approach: | They propose two attacks that aim to force the LLM ranker to prefer a specific passage and rank it at the top. |
| Outcome: | The proposed attacks aim to force the LLM ranker to prefer a specific passage and rank it at the top. |